I think it is worth noting that Ambitious Impact (formerly known as Charity Entrepreneurship) has jumped into the extremely competitive space of for-profit startups BEFORE trying to help build the AI Safety space.
Some quick thoughts/background:
1. The AI Safety space has LOADS of very smart people that can't get jobs because there aren't enough organisations to hire them. It might be the biggest bottleneck in the cause area. Meanwhile, capabilities literally has dozens of billions being thrown into it
2. For-profit entrepreneurship isn't in Ambitious Impact's (AI, lol) top cause area
3. I brought this up in the past, and Joey responded in this post. I think his post was overall a useful start, specific in some ways but vague in others. Vague in a 'hey maybe you should look into this but I won't tell you why' kinda way
Here is what I think is going on: there are people (maybe including Joey) 'in-the-know' about some things that make creating longtermist/AI safety startups really hard, but some of those reasons aren't being discussed publicly out of fear of shaming people for their failures and/or reluctance to put their money where their mouth is on x-risk.
I think we need a public discussion about whats going on here. Our lives may literally depend on it, even if Ambitious Impact doesn't think so.
Reactions
Curated and popular this week
·
This is a Draft Amnesty Week draft. It may not be polished, up to my usual standards, fully thought through, or fully fact-checked.
Commenting and feedback guidelines:
I'm posting this to get it out there. I'd love to see comments that take the ideas forward, but criticism of my argument won't be as useful at this time, in part because I won't do any further work on it.
This is a post I drafted in November 2023, then updated for an hour in March 2025. I don’t think I’ll ever finish it so I am just leaving it in this draft form for draft amnesty week (I know I'm late). I don’t think it is particularly well calibrated, but mainly just makes a bunch of points that I haven’t seen assembled elsewhere. Please take it as extremely low-confidence and there being a low-likelihood of this post describing these dynamics perfectly.
I’ve worked at both EA charities and non-EA charities, and the EA funding landscape is unlike any other I’ve ever been in. This can be good — funders are often willing to take high-risk, high-reward bets on projects that might otherwise never get funded, and the amount of friction for getting funding is significantly lower.
But, there is an orientation toward funders (and in particular staff at some major funders), that seems extremely unusual for charitable communities: a high degree of deference to their opinions.
As a reference, most other charitable communities I’ve worked in have viewed funders in a much more mixed light. Engaging with them is necessary, yes, but usually funders (including large, thoughtful foundations like Open Philanthropy) are viewed as… an unaligned third party who is instrumentally useful to your organization, but whose opinions on your work should hold relatively little or no weight, given that they are a non-expert on the direct work, and often have bad ideas about how to do what you are doing.
I think there are many good reasons to take funders’ perspectives seriously, and I mostly won’t cover these here. But, to
·
By Jim Chapman, Linkedin.
TL;DR: In 2023, I was a 57-year-old urban planning consultant and non-profit professional with 30 years of leadership experience. After talking with my son about rationality, effective altruism, and AI risks, I decided to pursue a pivot to existential risk reduction work. The last time I had to apply for a job was in 1994. By the end of 2024, I had spent ~740 hours on courses, conferences, meetings with ~140 people, and 21 job applications. I hope that by sharing my experiences, you can gain practical insights, inspiration, and resources to navigate your career transition, especially for those who are later in their career and interested in making an impact in similar fields. I share my experience in 5 sections - sparks, take stock, start, do, meta-learnings, and next steps. [Note - as of 03/05/2025, I am still pursuing my career shift.]
Sparks – 2022
During a Saturday bike ride, I admitted to my son, “No, I haven’t heard of effective altruism.” On another ride, I told him, “I'm glad you’re attending the EAGx Berkely conference." Some other time, I said, "Harry Potter and Methods of Rationality sounds interesting. I'll check it out." While playing table tennis, I asked, "What do you mean ChatGPT can't do math? No calculator? Next token prediction?" Around tax-filing time, I responded, "You really think retirement planning is out the window? That only 1 of 2 artificial intelligence futures occurs – humans flourish in a post-scarcity world or humans lose?" These conversations intrigued and concerned me.
After many more conversations about rationality, EA, AI risks, and being ready for something new and more impactful, I decided to pivot my career to address my growing concerns about existential risk, particularly AI-related. I am very grateful for those conversations because without them, I am highly confident I would not have spent the last year+ doing that.
Take Stock - 2023
I am very concerned about existential risk cause areas in ge
·
Written anonymously because I work in a field where there is a currently low but non-negligible and possibly high future risk of negative consequences for criticizing Trump and Trumpism.
This post is an attempt to cobble together some ideas about the current situation in the United States and its impact on EA. I invite discussion on this, not only from Americans, but also those with advocacy experience in countries that are not fully liberal democracies (especially those countries where state capacity is substantial and autocratic repression occurs).
I've deleted a lot of text from this post in various drafts because I find myself getting way too in the weeds discoursing on comparative authoritarian studies, disinformation and misinformation (this is a great intro, though already somewhat outdated), and the dangers of the GOP.[1] I will note that I worry there is still a tendency to view the administration as chaotic and clumsy but retaining some degree of good faith, which strikes me as quite naive.
For the sake of brevity and focus, I will take these two things to be true, and try to hypothesize what they mean for EA. I'm not going to pretend these are ironclad truths, but I'm fairly confident in them.[2]
1. Under Donald Trump, the Republican Party (GOP) is no longer substantially committed to democracy and the rule of law.
1. The GOP will almost certainly continue to engage in measures that test the limits of constitutional rule as long as Trump is alive, and likely after he dies.
2. The Democratic Party will remain constrained by institutional and coalition factors that prevent it from behaving like the GOP. That is, absent overwhelming electoral victories in 2024 and 2026 (and beyond), the Democrats' comparatively greater commitment to rule of law and democracy will prevent systematic purging of the GOP elites responsible for democratic backsliding; while we have not crossed the Rubicon yet, it will get much worse before things get better.
2. T
"As shown in this table 0% of CE staff (including me) identify AI as their top cause area. I think across the team people's reasons are varied but cluster around something close to epistemic scepticism. My personal perspective is also in line with that."
A quote from Joey replying to your last post. Why would you start an org around something none of your staff have as their top cause area? All CE charities to date have focused on global development or animal welfare, why would they switch focus to AI now? Doesn't seem so mysterious to me anyway.
CE incubated Training for Good, which runs two AI-related fellowships. They didn’t start out with an AI focus, but they also didn’t start out with a GHD or animal welfare focus.
AIM simply doesn't rate AI safety as a priority cause area. It's not any particular organisation's job to work on your favourite cause area. They are allowed to have a different prioritisation from you.
I think Yanni isn't writing about personal favourites. Assuming there is such a thing as objective truth, it makes sense to discuss cause prioritization as an objective question.
Hmmm, I think the fact that you felt this was worth pointing out AND that people upvoted it, means that I haven't made my point clear. My major concern is that there are things known about the challenges that come with incubating longtermist orgs that aren't being discussed openly.
Maybe I misunderstood you.
I think AIM doesn’t constitute evidence for this. Your top hypothesis should be that they don’t think AI safety is that good of a cause area, before positing the more complicated explanation. I say this partly based on interacting with people who have worked at AIM.
Just as the EA community does not own its donors' money -- one of the most upvoted posts ever -- it also doesn't own the financial sacrifices people at A.I. make to do the work they think is important. People who donate to, and work at, A.I. know that it has a neartermist focus.
Looking at funding trends over the past few years, it seems relatively easier for new/newish AI safety organizations to get supported than new/newish global health or animal advocacy organizations. For example, Redwood got over $20MM in funding from EA sources in the first ~2 years of its existence. Although the funding bar may be higher now than when those grants were made, I'm not convinced that the bottleneck here is that new AI safety orgs can't get the support needed to launch.
Sorry, it is so confusing to refer to AIM as 'A.I.', particularly in this context...
Yeah that was me attempting to be a bit cheeky but probably not worth it in exchange for clarity.
Is not enough organizations really the problem? For technical AI safety research, at least, I hear research management capacity is a bottleneck. A new technical AI safety org would compete with the others over the same potential research managers.
Another issue could be that few interventions seem net positive (maybe things have changed since that comment 3 years ago).
That's an interesting hypothesis. I think "seem" is an important word, because it points to me something I see as another issue - inaction leading from conservativeness, leading to capabilities pulling even further away.
TBH this is me putting my tin foil hat on a bit, but even if my most paranoid thoughts are ruled out, it is still a weirdly under-discussed issue in the space and I'm cashing in all my chips for the Amnesty Week thing. Yolo.